Climate Ocean Modeling on Parallel Computers

Ocean modeling plays an important role in both understanding the current climatic conditions and predicting future climate change. However, modeling the ocean circulation at various spatial and temporal scales is a very challenging computational task. In contrast to the atmosphere, where the dominant weather system has a spatial scale of 1000s km, much of the ocean energy is associated with mesoscale eddies (equivalent to the storms in the atmosphere) with a spatial scale of 100s km near the equator to 10s km at high latitudes. With an order of magnitude smaller in both longitudinal and latitudinal directions and a few time shorter in the temporal scale, a minimum ocean model is at least 100 times more computationally demanding than a typical atmospheric model. Implementing a well-designed parallel ocean code and improving the computational efficiency of the ocean model will significantly reduce the total research time to complete these studies. There are many challenges to design an efficient ocean modeling code on parallel systems, such as how to deal with an irregular computing geometry on a parallel system and how to port a code from one system to other systems. This chapter reports on our efforts to implement an efficient parallel ocean model on distributed memory and shared memory systems. The computation domains of an ocean model is usually irregular. A simple method to deal with irregular geometries is to use a rectangular geometry to approach the irregular domain. In this case, many processors will be idle on land while other processors are working on the ocean domain. For some applications, it will cause more than 30% processors to be idle. We have developed a flexible partitioning technique for irregular ocean geometries on parallel systems. It has 2D partitioning features and works on any irregular geometry, and the communication pattern on the subdomains has been designed as a virtual torus. MPI software is used for communication which is required when subdomains on each processor need neighboring boundary data information. This partitioning structure has dramatically saved computing resources, leading to a significant speed up. Because of the portability of this software, the code can be executed on any parallel system which supports MPI library. We want to emphasize the portability of the ocean model across a variety of parallel platforms, ranging from the most powerful supercomputers to the affordable desktop parallel PC cluster (Beowulf-class system). To this end, we have successfully ported the most widely used three-dimensional time-dependent ocean general circulation models to various parallel systems, including the scalable parallel systems Cray T3E-600, the shared memory system HP Exemplar SPP-2000, and the 16-node PC cluster Beowulf system. Procedures on how to implement and optimize the code to different system are discussed, and intensive comparisons of wallclock time with various grid sizes are made among several parallel systems. Scientific results fiom a Atlantic ocean model with high resolutions have been obtained using 256 processors on the Cray T3D.